Universal Access To All Knowledge
Home Donate | Store | Blog | FAQ | Jobs | Volunteer Positions | Contact | Bios | Forums | Projects | Terms, Privacy, & Copyright
Search: Advanced Search
Anonymous User (login or join us)
Upload

Reply to this post | See parent post | Go Back
View Post [edit]

Poster: Administrator, Curator, or Staffh4ck3rm1k3 Date: Dec 22, 2009 9:51am
Forum: forums Subject: Re: Checking out items via FTP?

Sure, here is the mail I sent :
Hi there,

I have started to produce openstreetmap files for the archive.

For example converted data from the EPA and census buro.

http://www.archive.org/details/ZCTA_NJ
http://www.archive.org/details/OpenstreetmapZipcodes
http://www.archive.org/details/EPADatapointsOpenStreetmap


My problem is that I have lots of small files, and would like to
update them with FTP.

Also, what I really need is some form of GIT/BZR/HG distributed
version control system for editing these files.
they will be updated on a regular basis.

The data is publicdomain/creative commons, but archive.org is not
ideal for processing them.

Are you interested in working with me to making a better way to host
map data on archive org?
I am a programmer and would like to work with you on this.
thanks,
mike

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or Stafftracey pooh Date: Feb 18, 2010 2:02pm
Forum: forums Subject: Re: Checking out items via FTP?

and once again, lots of small files = fail with our site.

we have some interesting tools/solutions for that,
including an automatic and efficient 'zipview' utility
that we could try to setup for your items.
this allows a single file to be pulled out a of zip to an end user.
--t

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or Stafftracey pooh Date: Feb 18, 2010 2:01pm
Forum: forums Subject: Re: Checking out items via FTP?

the only robo option for something like that that we have available is S3.
that would allow for all of what you want i think..
-t

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or Staffh4ck3rm1k3 Date: Dec 22, 2009 9:56am
Forum: forums Subject: Re: Checking out items via FTP?

The other thing that I would like to say is that for these zip code files we need to have small individual files, because people will only access one or two at a given time.
To put them all into a zip file would defeat the purpose, unless there is a way to access only a certain offset in the file some how.